A digital image represents a visual scene as an array of pixels. Each pixel stores the image intensity level for a specific point in the visual scene. For example, in a black-and-white digital image, each pixel may contain an 8-bit value that expresses a luminance level as a number between 0 and 255, with 0 representing black and 255 representing white. In a color digital image, each pixel may have multiple components or color planes, each component representing the intensity level for a “color” (e.g., RGB (Red-Green-Blue) or CMY (Cyan-Magenta-Yellow) color planes) or a luminance/chrominance value.
Many digital images are captured in such a way that the dynamic range available in the pixel format is underutilized. For instance, in a given 8-bit image, most of the image intensity information may cluster in a small sub-range of the available 0-to-255 range. This results in a displayable image with many image features that cannot be readily discerned by a human viewer, as the contrast sensitivity function (CSF) of the human visual system cannot resolve such small differences in image intensity level.
Fine-contrast features in a digital image can be made more discernable by application of a contrast-enhancement algorithm known as histogram equalization (a histogram indicates the relative occurrence of each possible pixel value in a particular digital image—the magnitude of each bin in the histogram represents the frequency of occurrence of a corresponding pixel value). A short description of histogram equalization (HE) follows.
FIG. 1 shows a hypothetical histogram H(x) for an input image (actual histograms are rarely smooth functions as shown, and are often multimodal). Although the dynamic range of the pixel format allows for intensity levels x between 0 and L-1, H(x) indicates that the hypothetical image only occupies values between xMIN and xMAX.
Histogram equalization attempts to redistribute the various image intensity levels to new levels so that the histogram E(y) of the output image better approximates a uniform distribution over the entire range 0 to L-1. A remapping function y=F(x) is calculated such thatE(y)=E(F(x))=H(F−1(y))approximates such a uniform distribution. Although E(y) in FIG. 1 is shown as perfectly uniform, it is noted that with quantized data as exists in histogram bins, HE will usually not result in L bins with an equal number of pixels in each. Nevertheless, the HE algorithm distributes the quantized data in as uniform of a manner as is possible (see FIG. 14 for actual examples).
With a near-symmetric distribution such as H(x), the mean of the distribution xM will map approximately to the level L/2 in E(y). When the distribution is not symmetric, the specific location that xM will map to depends on how the histogram is skewed. These characteristics of HE can cause problems when HE is applied to a video sequence—although each output frame may have excellent contrast, the mean value is not preserved. Thus a histogram-equalized video sequence may have undesirable image flicker artifacts. Also, since the mean may change significantly, scenes which are intended to be dark (or bright) may no longer appear so.
One researcher has proposed a different HE scheme to combat the problems inherent in applying classical histogram equalization to a video sequence. “Contrast Enhancement Using Brightness Preserving Bi-Histogram Equalization”, Y.-T. Kim, IEEE Transactions on Consumer Electronics, Vol. 43, No. 1, February 1997; see also Kim & Cho, “Image Enhancing Method Using Mean-Separate Histogram Equalization and a Circuit Therefor”, U.S. Pat. No. 5,963,665, Oct. 5, 1999.
Referring to FIG. 2, the intended operation of Kim's method is illustrated. Kim calculates the mean intensity level xM for the input image. Kim then ensures that xM maps to itself in the output image by dividing the image into two groups of pixels, those above xM and those below this value. Each group is then histogram-equalized separately, i.e., the lower group of pixels is equalized over the range 0 to xM, and the upper group of pixels is equalized over the range xM to L−1. This results in an output histogram approximately represented by B(y) in FIG. 2. Kim notes that for a symmetric distribution H(x), the mean output intensity achieved with his mapping is (xM+L/2)/2, i.e., the mean shifts halfway towards the middle intensity, instead of all the way to the middle intensity like in classical HE. Kim claims that this results in less brightness shifting over an image sequence. The Kim approach will be referred to herein as BBHE (Brightness-preserving Bi-Histogram Equalization).
It is recognized herein that BBHE can still produce undesirable artifacts when applied to a video sequence. One problem with BBHE is that many different image distributions can have the same mean. Particularly when these distributions are multi-modal, some regions of the image may experience drastic shifts in intensity even though the mean intensity is preserved. Another problem is that the mean statistic is extremely sensitive to outliers, i.e., a relatively few pixels with intensities located far from the mean can have a profound influence on mean intensity. Thus when the histogram is not symmetric about the image mean, large shifts in mean brightness can still occur, and substantial brightness shifts can still occur for some ranges of pixel values.
These effects are illustrated by the simplified bimodal distributions of FIGS. 3 and 4. In FIG. 3, the input histogram is divided into two modes; mode 20 contains 25% of the pixels, and is uniformly distributed over a range 0 to 0.2 L. Mode 22 contains the other 75% of the pixels, and is uniformly distributed over a range 0.25 L to 0.35 L. The input image mean xM=0.25 L. With this input, BBHE produces a result equivalent to that produced with mapping function 24, and the output histogram is (somewhat coincidentally) a uniform distribution 26. Note that the mean level has shifted all the way up to L/2, i.e., it has doubled. And mode 22, which contains 75% of the pixels, has been spread by a factor of 7.5.
Now compare this result with FIG. 4, which has the same mean xM=0.25 L. In FIG. 4, the input histogram is also divided into two modes; mode 28 contains ⅚ths of the pixels, and is uniformly distributed over a range 0.05 L to 0.2 L. Mode 30 contains the remaining ⅙ of the pixels, and is uniformly distributed over a range 0.75 L to L. When BBHE is applied to this input, distribution 34 results, and the mean level has shifted down to ˜0.208 L. Mode 30, which represented very bright pixels in the scene, has been shifted such that some of its pixels now appear very dark. These simple examples show that with BBHE, the input scene, as well as the average brightness, can at the BBHE output change significantly and in many different ways, even though the input mean value is stabilized.